Fix type hints for txredis. #12042

clokep · 2022-02-18T21:07:34Z

The type hints were subtly wrong for txredis. A bunch of places it stated we used RedisProtocol, but it was actually a ConnectionHandler... this is made more confusing that ConnectionHandler essentially choosing connections from the factory's pool of connections. There's also a ReplicationCommandHandler, which is a different thing specific to Synapse.

You end up with something like this (with lots of things referring to each other...). Fat lines are sub-classes, thin lines are references. (Note that the pool below can have multiple RedisSubscriber instances, I put one for clarity and because we set poolsize=1).

flowchart TD
RedisDirectTcpReplicationClientFactory ==> SynapseRedisFactory
RedisDirectTcpReplicationClientFactory -->|synapse_outbound_redis_connection| ConnectionHandler
RedisDirectTcpReplicationClientFactory -->|synapse_handler| ReplicationCommandHandler 
SynapseRedisFactory ==> RedisFactory
ConnectionHandler -->|_factory| RedisFactory
RedisFactory --> |handler| ConnectionHandler
RedisFactory --> |pool| RedisSubscriber
RedisSubscriber -->|synapse_outbound_redis_connection| ConnectionHandler
RedisSubscriber -->|synapse_handler| ReplicationCommandHandler
RedisSubscriber ==> SubscriberProtocol
SubscriberProtocol ==> RedisProtocol

clokep · 2022-02-18T21:11:43Z

Something that's unclear to me -- we seem to be (sometimes?) sharing a ConnectionHandler across factories, I suspect that's wrong and they're meant to be one-to-one.

clokep · 2022-02-22T19:32:26Z

Something that's unclear to me -- we seem to be (sometimes?) sharing a ConnectionHandler across factories, I suspect that's wrong and they're meant to be one-to-one.

@erikjohnston Any thoughts on this? Is ConnectionHandler meant to be per-factory or a singleton used everywhere? We seem to be doing weird things with this.

erikjohnston · 2022-02-23T10:04:59Z

What do you mean by sharing?

Something that's unclear to me -- we seem to be (sometimes?) sharing a ConnectionHandler across factories, I suspect that's wrong and they're meant to be one-to-one.

@erikjohnston Any thoughts on this? Is ConnectionHandler meant to be per-factory or a singleton used everywhere? We seem to be doing weird things with this.

Where are you seeing that? I'm not seeing anywhere in our code that we instansiate ConnectionHandler, we just pass around the type?

clokep · 2022-02-23T18:14:06Z

Where are you seeing that? I'm not seeing anywhere in our code that we instansiate ConnectionHandler, we just pass around the type?

It is a bit confusing, but RedisDirectTcpReplicationClientFactory ends with two ConnectionHandlers, one of which might be unused (?):

synapse/synapse/replication/tcp/redis.py

Lines 315 to 331 in d0e78af

    
           def __init__( 
        
               self, hs: "HomeServer", outbound_redis_connection: txredisapi.RedisProtocol 
        
           ): 
        
               super().__init__( 
        
                   hs, 
        
                   uuid="subscriber", 
        
                   dbid=None, 
        
                   poolsize=1, 
        
                   replyTimeout=30, 
        
                   password=hs.config.redis.redis_password, 
        
               ) 
        
               self.synapse_handler = hs.get_tcp_replication() 
        
               self.synapse_stream_name = hs.hostname 
        
               self.synapse_outbound_redis_connection = outbound_redis_connection

The call to super().__Init__(...) (which goes to txredis) instantiates a handler:

https://github.com/IlyaSkriblovsky/txredisapi/blob/c89a49829d1c48baa10d03c41d39c261c02ad6e1/txredisapi.py#L2289

But we also store one as synapse_outbound_redis_connection. This then gets stored onto each RedisSubscriber, but RedisSubscriber seems to only ever use synapse_outbound_redis_connection. Mostly I'm just confused at what happens to the other handler, does it just sit around doing nothing? It seems to get passed into a Deferred that we don't seem to care about?

erikjohnston · 2022-02-24T09:52:43Z

Ah right, yes. So at any given time there are two connections to Redis. One is dedicated to receiving pub/sub messages (i.e. the connection that does SUBSCRIBE, which takes over the connection), and one for sending all other commands. The synapse_outbound_redis_connection is the latter, basically, and so gets passed around to anywhere that might need to send a command to Redis.

clokep · 2022-03-07T13:39:53Z

Ah right, yes. So at any given time there are two connections to Redis. One is dedicated to receiving pub/sub messages (i.e. the connection that does SUBSCRIBE, which takes over the connection), and one for sending all other commands. The synapse_outbound_redis_connection is the latter, basically, and so gets passed around to anywhere that might need to send a command to Redis.

Right! Well, that's confusing, but regardless I think this PR makes the type hints more accurate?

@IronTooch

Synapse 1.56.0 (2022-04-05) =========================== Synapse will now refuse to start up if open registration is enabled, in order to help mitigate abuse across the federation. If you would like to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285), [recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html) or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification in order to prevent automated registration from bad actors. This check can be disabled by setting the `enable_registration_without_verification` option in your homeserver configuration file to `true`. More details are available in the [upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default). Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the [upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale) for details. Internal Changes ---------------- - Bump the version of `black` for compatibility with the latest `click` release. ([\#12320](matrix-org/synapse#12320)) Synapse 1.56.0rc1 (2022-03-29) ============================== Features -------- - Allow modules to store already existing 3PID associations. ([\#12195](matrix-org/synapse#12195)) - Allow registering server administrators using the module API. Contributed by Famedly. ([\#12250](matrix-org/synapse#12250)) Bugfixes -------- - Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](matrix-org/synapse#12087)) - Fix a long-standing bug where events from ignored users would still be considered for relations. ([\#12227](matrix-org/synapse#12227), [\#12232](matrix-org/synapse#12232), [\#12285](matrix-org/synapse#12285)) - Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](matrix-org/synapse#12228)) - Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](matrix-org/synapse#12261)) Improved Documentation ---------------------- - Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](matrix-org/synapse#12231)) - Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](matrix-org/synapse#12243)) - Correct `check_username_for_spam` annotations and docs. ([\#12246](matrix-org/synapse#12246)) - Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\#12275](matrix-org/synapse#12275)) - HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](matrix-org/synapse#12279)) Internal Changes ---------------- - Rename `shared_rooms` to `mutual_rooms` ([MSC2666](matrix-org/matrix-spec-proposals#2666)), as per proposal changes. ([\#12036](matrix-org/synapse#12036)) - Remove check on `update_user_directory` for shared rooms handler ([MSC2666](matrix-org/matrix-spec-proposals#2666)), and update/expand documentation. ([\#12038](matrix-org/synapse#12038)) - Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](matrix-org/matrix-spec-proposals#2716) instead of abusing `auth_event_ids`. ([\#12083](matrix-org/synapse#12083), [\#12304](matrix-org/synapse#12304)) - Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](matrix-org/synapse#12091), [\#12322](matrix-org/synapse#12322)) - Add tests for database transaction callbacks. ([\#12198](matrix-org/synapse#12198)) - Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](matrix-org/synapse#12199)) - Add missing type hints for cache storage. ([\#12216](matrix-org/synapse#12216)) - Add missing type hints for storage. ([\#12248](matrix-org/synapse#12248), [\#12255](matrix-org/synapse#12255)) - Add type hints to tests files. ([\#12224](matrix-org/synapse#12224), [\#12240](matrix-org/synapse#12240), [\#12256](matrix-org/synapse#12256)) - Use type stubs for `psycopg2`. ([\#12269](matrix-org/synapse#12269)) - Improve type annotations for `execute_values`. ([\#12311](matrix-org/synapse#12311)) - Clean-up logic around rebasing URLs for URL image previews. ([\#12219](matrix-org/synapse#12219)) - Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](matrix-org/synapse#12225)) - Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](matrix-org/synapse#12237)) - Generate announcement links in the release script. ([\#12242](matrix-org/synapse#12242)) - Improve error message when dependencies check finds a broken installation. ([\#12244](matrix-org/synapse#12244)) - Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](matrix-org/synapse#12258)) - Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\#12262](matrix-org/synapse#12262), [\#12288](matrix-org/synapse#12288)) - Optionally include account validity expiration information to experimental [MSC3720](matrix-org/matrix-spec-proposals#3720) account status responses. ([\#12266](matrix-org/synapse#12266)) - Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](matrix-org/synapse#12272)) - Re-enable Complement concurrency in CI. ([\#12283](matrix-org/synapse#12283)) - Remove unused test utilities. ([\#12291](matrix-org/synapse#12291)) - Enhance logging for inbound federation events. ([\#12301](matrix-org/synapse#12301)) - Fix compatibility with the recently-released Jinja 3.1. ([\#12313](matrix-org/synapse#12313)) - Avoid trying to calculate the state at outlier events. ([\#12314](matrix-org/synapse#12314)) Synapse 1.55.2 (2022-03-24) =========================== This patch version reverts the earlier fixes from Synapse 1.55.1, which could cause problems in certain deployments, and instead adds a cap to the version of Jinja to be installed. Again, this is to fix an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, and again, deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected. Internal Changes ---------------- - Pin Jinja to <3.1.0, as Synapse fails to start with Jinja 3.1.0. ([\#12297](matrix-org/synapse#12297)) - Revert changes from 1.55.1 as they caused problems with older versions of Jinja ([\#12296](matrix-org/synapse#12296)) Synapse 1.55.1 (2022-03-24) =========================== This is a patch release that fixes an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, released on March 24th, 2022. Deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected. Internal Changes ---------------- - Remove uses of the long-deprecated `jinja2.Markup` which would prevent Synapse from starting with Jinja 3.1.0 or above installed. ([\#12289](matrix-org/synapse#12289)) Synapse 1.55.0 (2022-03-22) =========================== This release removes a workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. **This breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700; Mjolnir users should upgrade Mjolnir before upgrading Synapse to this version. This release also moves the location of the `synctl` script; see the [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved) for more details. Internal Changes ---------------- - Tweak copy for default Single Sign-On account details template to better adhere to mobile app store guidelines. ([\#12265](matrix-org/synapse#12265), [\#12260](matrix-org/synapse#12260)) Synapse 1.55.0rc1 (2022-03-15) ============================== Features -------- - Add third-party rules callbacks `check_can_shutdown_room` and `check_can_deactivate_user`. ([\#12028](matrix-org/synapse#12028)) - Improve performance of logging in for large accounts. ([\#12132](matrix-org/synapse#12132)) - Add experimental env var `SYNAPSE_ASYNC_IO_REACTOR` that causes Synapse to use the asyncio reactor for Twisted. ([\#12135](matrix-org/synapse#12135)) - Support the stable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440): threads. ([\#12151](matrix-org/synapse#12151)) - Add a new Jinja2 template filter to extract the local part of an email address. ([\#12212](matrix-org/synapse#12212)) Bugfixes -------- - Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse v1.48.0. ([\#12090](matrix-org/synapse#12090)) - Fix a long-standing bug when redacting events with relations. ([\#12113](matrix-org/synapse#12113), [\#12121](matrix-org/synapse#12121), [\#12130](matrix-org/synapse#12130), [\#12189](matrix-org/synapse#12189)) - Fix a bug introduced in Synapse 1.7.2 whereby background updates are never run with the default background batch size. ([\#12157](matrix-org/synapse#12157)) - Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse v1.41.0. ([\#12175](matrix-org/synapse#12175)) - Fix a bug introduced in Synapse 1.54.0 that broke background updates on sqlite homeservers while search was disabled. ([\#12215](matrix-org/synapse#12215)) - Fix a long-standing bug when a `filter` argument with `event_fields` which did not include the `unsigned` field could result in a 500 error on `/sync`. ([\#12234](matrix-org/synapse#12234)) Improved Documentation ---------------------- - Fix complexity checking config example in [Resource Constrained Devices](https://matrix-org.github.io/synapse/v1.54/other/running_synapse_on_single_board_computers.html) docs page. ([\#11998](matrix-org/synapse#11998)) - Improve documentation for demo scripts. ([\#12143](matrix-org/synapse#12143)) - Updates to the Room DAG concepts development document. ([\#12179](matrix-org/synapse#12179)) - Document that the `typing`, `to_device`, `account_data`, `receipts`, and `presence` stream writer can only be used on a single worker. ([\#12196](matrix-org/synapse#12196)) - Document that contributors can sign off privately by email. ([\#12204](matrix-org/synapse#12204)) Deprecations and Removals ------------------------- - **Remove workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. Breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700 - **`synctl` has been moved into into `synapse._scripts` and is exposed as an entry point; see [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved). ([\#12140](matrix-org/synapse#12140)) - Remove backwards compatibilty with pagination tokens from the `/relations` and `/aggregations` endpoints generated from Synapse < v1.52.0. ([\#12138](matrix-org/synapse#12138)) - The groups/communities feature in Synapse has been deprecated. ([\#12200](matrix-org/synapse#12200)) Internal Changes ---------------- - Simplify the `ApplicationService` class' set of public methods related to interest checking. ([\#11915](matrix-org/synapse#11915)) - Add config settings for background update parameters. ([\#11980](matrix-org/synapse#11980)) - Correct type hints for txredis. ([\#12042](matrix-org/synapse#12042)) - Limit the size of `aggregation_key` on annotations. ([\#12101](matrix-org/synapse#12101)) - Add type hints to tests files. ([\#12108](matrix-org/synapse#12108), [\#12146](matrix-org/synapse#12146), [\#12207](matrix-org/synapse#12207), [\#12208](matrix-org/synapse#12208)) - Move scripts to Synapse package and expose as setuptools entry points. ([\#12118](matrix-org/synapse#12118)) - Add support for cancellation to `ReadWriteLock`. ([\#12120](matrix-org/synapse#12120)) - Fix data validation to compare to lists, not sequences. ([\#12128](matrix-org/synapse#12128)) - Fix CI not attaching source distributions and wheels to the GitHub releases. ([\#12131](matrix-org/synapse#12131)) - Remove unused mocks from `test_typing`. ([\#12136](matrix-org/synapse#12136)) - Give `scripts-dev` scripts suffixes for neater CI config. ([\#12137](matrix-org/synapse#12137)) - Move the snapcraft configuration file to `contrib`. ([\#12142](matrix-org/synapse#12142)) - Enable [MSC3030](matrix-org/matrix-spec-proposals#3030) Complement tests in CI. ([\#12144](matrix-org/synapse#12144)) - Enable [MSC2716](matrix-org/matrix-spec-proposals#2716) Complement tests in CI. ([\#12145](matrix-org/synapse#12145)) - Add test for `ObservableDeferred`'s cancellation behaviour. ([\#12149](matrix-org/synapse#12149)) - Use `ParamSpec` in type hints for `synapse.logging.context`. ([\#12150](matrix-org/synapse#12150)) - Prune unused jobs from `tox` config. ([\#12152](matrix-org/synapse#12152)) - Move CI checks out of tox, to facilitate a move to using poetry. ([\#12153](matrix-org/synapse#12153)) - Avoid generating state groups for local out-of-band leaves. ([\#12154](matrix-org/synapse#12154)) - Avoid trying to calculate the state at outlier events. ([\#12155](matrix-org/synapse#12155), [\#12173](matrix-org/synapse#12173), [\#12202](matrix-org/synapse#12202)) - Fix some type annotations. ([\#12156](matrix-org/synapse#12156)) - Add type hints for `ObservableDeferred` attributes. ([\#12159](matrix-org/synapse#12159)) - Use a prebuilt Action for the `tests-done` CI job. ([\#12161](matrix-org/synapse#12161)) - Reduce number of DB queries made during processing of `/sync`. ([\#12163](matrix-org/synapse#12163)) - Add `delay_cancellation` utility function, which behaves like `stop_cancellation` but waits until the original `Deferred` resolves before raising a `CancelledError`. ([\#12180](matrix-org/synapse#12180)) - Retry HTTP replication failures, this should prevent 502's when restarting stateful workers (main, event persisters, stream writers). Contributed by Nick @ Beeper. ([\#12182](matrix-org/synapse#12182)) - Add cancellation support to `@cached` and `@cachedList` decorators. ([\#12183](matrix-org/synapse#12183)) - Remove unused variables. ([\#12187](matrix-org/synapse#12187)) - Add combined test for HTTP pusher and push rule. Contributed by Nick @ Beeper. ([\#12188](matrix-org/synapse#12188)) - Rename `HomeServer.get_tcp_replication` to `get_replication_command_handler`. ([\#12192](matrix-org/synapse#12192)) - Remove some dead code. ([\#12197](matrix-org/synapse#12197)) - Fix a misleading comment in the function `check_event_for_spam`. ([\#12203](matrix-org/synapse#12203)) - Remove unnecessary `pass` statements. ([\#12206](matrix-org/synapse#12206)) - Update the SSO username picker template to comply with SIWA guidelines. ([\#12210](matrix-org/synapse#12210)) - Improve code documentation for the typing stream over replication. ([\#12211](matrix-org/synapse#12211))

Synapse 1.55.0 (2022-03-22) =========================== This release removes a workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. **This breaks compatibility with Mjolnir 1.3.1 and earlier. ([\matrix-org#11700](matrix-org#11700; Mjolnir users should upgrade Mjolnir before upgrading Synapse to this version. This release also moves the location of the `synctl` script; see the [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved) for more details. Internal Changes ---------------- - Tweak copy for default Single Sign-On account details template to better adhere to mobile app store guidelines. ([\matrix-org#12265](matrix-org#12265), [\matrix-org#12260](matrix-org#12260)) Synapse 1.55.0rc1 (2022-03-15) ============================== Features -------- - Add third-party rules callbacks `check_can_shutdown_room` and `check_can_deactivate_user`. ([\matrix-org#12028](matrix-org#12028)) - Improve performance of logging in for large accounts. ([\matrix-org#12132](matrix-org#12132)) - Add experimental env var `SYNAPSE_ASYNC_IO_REACTOR` that causes Synapse to use the asyncio reactor for Twisted. ([\matrix-org#12135](matrix-org#12135)) - Support the stable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440): threads. ([\matrix-org#12151](matrix-org#12151)) - Add a new Jinja2 template filter to extract the local part of an email address. ([\matrix-org#12212](matrix-org#12212)) Bugfixes -------- - Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse v1.48.0. ([\matrix-org#12090](matrix-org#12090)) - Fix a long-standing bug when redacting events with relations. ([\matrix-org#12113](matrix-org#12113), [\matrix-org#12121](matrix-org#12121), [\matrix-org#12130](matrix-org#12130), [\matrix-org#12189](matrix-org#12189)) - Fix a bug introduced in Synapse 1.7.2 whereby background updates are never run with the default background batch size. ([\matrix-org#12157](matrix-org#12157)) - Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse v1.41.0. ([\matrix-org#12175](matrix-org#12175)) - Fix a bug introduced in Synapse 1.54.0 that broke background updates on sqlite homeservers while search was disabled. ([\matrix-org#12215](matrix-org#12215)) - Fix a long-standing bug when a `filter` argument with `event_fields` which did not include the `unsigned` field could result in a 500 error on `/sync`. ([\matrix-org#12234](matrix-org#12234)) Improved Documentation ---------------------- - Fix complexity checking config example in [Resource Constrained Devices](https://matrix-org.github.io/synapse/v1.54/other/running_synapse_on_single_board_computers.html) docs page. ([\matrix-org#11998](matrix-org#11998)) - Improve documentation for demo scripts. ([\matrix-org#12143](matrix-org#12143)) - Updates to the Room DAG concepts development document. ([\matrix-org#12179](matrix-org#12179)) - Document that the `typing`, `to_device`, `account_data`, `receipts`, and `presence` stream writer can only be used on a single worker. ([\matrix-org#12196](matrix-org#12196)) - Document that contributors can sign off privately by email. ([\matrix-org#12204](matrix-org#12204)) Deprecations and Removals ------------------------- - **Remove workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. Breaks compatibility with Mjolnir 1.3.1 and earlier. ([\matrix-org#11700](matrix-org#11700 - **`synctl` has been moved into into `synapse._scripts` and is exposed as an entry point; see [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved). ([\matrix-org#12140](matrix-org#12140)) - Remove backwards compatibilty with pagination tokens from the `/relations` and `/aggregations` endpoints generated from Synapse < v1.52.0. ([\matrix-org#12138](matrix-org#12138)) - The groups/communities feature in Synapse has been deprecated. ([\matrix-org#12200](matrix-org#12200)) Internal Changes ---------------- - Simplify the `ApplicationService` class' set of public methods related to interest checking. ([\matrix-org#11915](matrix-org#11915)) - Add config settings for background update parameters. ([\matrix-org#11980](matrix-org#11980)) - Correct type hints for txredis. ([\matrix-org#12042](matrix-org#12042)) - Limit the size of `aggregation_key` on annotations. ([\matrix-org#12101](matrix-org#12101)) - Add type hints to tests files. ([\matrix-org#12108](matrix-org#12108), [\matrix-org#12146](matrix-org#12146), [\matrix-org#12207](matrix-org#12207), [\matrix-org#12208](matrix-org#12208)) - Move scripts to Synapse package and expose as setuptools entry points. ([\matrix-org#12118](matrix-org#12118)) - Add support for cancellation to `ReadWriteLock`. ([\matrix-org#12120](matrix-org#12120)) - Fix data validation to compare to lists, not sequences. ([\matrix-org#12128](matrix-org#12128)) - Fix CI not attaching source distributions and wheels to the GitHub releases. ([\matrix-org#12131](matrix-org#12131)) - Remove unused mocks from `test_typing`. ([\matrix-org#12136](matrix-org#12136)) - Give `scripts-dev` scripts suffixes for neater CI config. ([\matrix-org#12137](matrix-org#12137)) - Move the snapcraft configuration file to `contrib`. ([\matrix-org#12142](matrix-org#12142)) - Enable [MSC3030](matrix-org/matrix-spec-proposals#3030) Complement tests in CI. ([\matrix-org#12144](matrix-org#12144)) - Enable [MSC2716](matrix-org/matrix-spec-proposals#2716) Complement tests in CI. ([\matrix-org#12145](matrix-org#12145)) - Add test for `ObservableDeferred`'s cancellation behaviour. ([\matrix-org#12149](matrix-org#12149)) - Use `ParamSpec` in type hints for `synapse.logging.context`. ([\matrix-org#12150](matrix-org#12150)) - Prune unused jobs from `tox` config. ([\matrix-org#12152](matrix-org#12152)) - Move CI checks out of tox, to facilitate a move to using poetry. ([\matrix-org#12153](matrix-org#12153)) - Avoid generating state groups for local out-of-band leaves. ([\matrix-org#12154](matrix-org#12154)) - Avoid trying to calculate the state at outlier events. ([\matrix-org#12155](matrix-org#12155), [\matrix-org#12173](matrix-org#12173), [\matrix-org#12202](matrix-org#12202)) - Fix some type annotations. ([\matrix-org#12156](matrix-org#12156)) - Add type hints for `ObservableDeferred` attributes. ([\matrix-org#12159](matrix-org#12159)) - Use a prebuilt Action for the `tests-done` CI job. ([\matrix-org#12161](matrix-org#12161)) - Reduce number of DB queries made during processing of `/sync`. ([\matrix-org#12163](matrix-org#12163)) - Add `delay_cancellation` utility function, which behaves like `stop_cancellation` but waits until the original `Deferred` resolves before raising a `CancelledError`. ([\matrix-org#12180](matrix-org#12180)) - Retry HTTP replication failures, this should prevent 502's when restarting stateful workers (main, event persisters, stream writers). Contributed by Nick @ Beeper. ([\matrix-org#12182](matrix-org#12182)) - Add cancellation support to `@cached` and `@cachedList` decorators. ([\matrix-org#12183](matrix-org#12183)) - Remove unused variables. ([\matrix-org#12187](matrix-org#12187)) - Add combined test for HTTP pusher and push rule. Contributed by Nick @ Beeper. ([\matrix-org#12188](matrix-org#12188)) - Rename `HomeServer.get_tcp_replication` to `get_replication_command_handler`. ([\matrix-org#12192](matrix-org#12192)) - Remove some dead code. ([\matrix-org#12197](matrix-org#12197)) - Fix a misleading comment in the function `check_event_for_spam`. ([\matrix-org#12203](matrix-org#12203)) - Remove unnecessary `pass` statements. ([\matrix-org#12206](matrix-org#12206)) - Update the SSO username picker template to comply with SIWA guidelines. ([\matrix-org#12210](matrix-org#12210)) - Improve code documentation for the typing stream over replication. ([\matrix-org#12211](matrix-org#12211)) # -----BEGIN PGP SIGNATURE----- # # iQIzBAABCAAdFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAmI53SwACgkQiISIDS7+ # X/RlgQ/9HX3moAXQw67uIII5BnSGAYdgovL2uM990urQ6HVYRxcnidRlWQUyqlJ0 # qbvpOrJXRpK5MvoFyrJg+ryJLEncoUiccs5WQHNt4t22zmUXusLH/Bxxc/U24s5D # 3gQVhYdjiyyR0dIzr/EwZsYGSgl9wANoKVJXbRInrE7gdMP5RT/IcifWBZETNWjQ # zPGp3R5RDYzFecDiZUcGJCldXC+WPOw0YlhMdR7Rm5NkiE6eM1kAREH+K5vxBM7i # H+nP//NLl+wL4nxb3f3paEgCakPkB+xuC6bZZ3NA7LfMw2OKBHdUqEl4rmXr4vCS # lboF9kDhb42uIKgVPBnVcf3hv4m5NbVA7f6k3tpFJOvqk9G58UhnqcjdXS8DBso6 # gQpk7Zk6ILU0j8i7N21R/aZLSwmQ96h1O3lNO+43usPbh6WjqMIHR/kv/ADyZEhZ # /6taXiOHtg7KaqtpHAy2UVuWjJ8c3a0vHbReRe77S+yMD7iggHbBTpSNMfgcM4yi # gimKq55Z0qhPLI7vjr4IOB5irEd64ciKnUI3IQfkyFoCMyJHo0fw7+boC/Kc5FgM # O7V69r+CBfJ9SsGeSlheiCTv6fGyh4n960ya2BQY+BHDjTFW+BUkTyjXkIeiIJ04 # CihTgluISePbrdRKL4JkceL+Im6NmpJhTOe1+Pjsncy/8WkOeos= # =HvNr # -----END PGP SIGNATURE----- # gpg: Signature made Tue Mar 22 14:29:00 2022 GMT # gpg: using RSA key 8101B7D59DF5ECDAD232DD108884880D2EFE5FF4 # gpg: Can't check signature: No public key # Conflicts: # synapse/appservice/__init__.py # synapse/handlers/pagination.py # synapse/push/baserules.py

Synapse 1.55.0 (2022-03-22) =========================== This release removes a workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. **This breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700; Mjolnir users should upgrade Mjolnir before upgrading Synapse to this version. This release also moves the location of the `synctl` script; see the [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved) for more details. Internal Changes ---------------- - Tweak copy for default Single Sign-On account details template to better adhere to mobile app store guidelines. ([\#12265](matrix-org/synapse#12265), [\#12260](matrix-org/synapse#12260)) Synapse 1.55.0rc1 (2022-03-15) ============================== Features -------- - Add third-party rules callbacks `check_can_shutdown_room` and `check_can_deactivate_user`. ([\#12028](matrix-org/synapse#12028)) - Improve performance of logging in for large accounts. ([\#12132](matrix-org/synapse#12132)) - Add experimental env var `SYNAPSE_ASYNC_IO_REACTOR` that causes Synapse to use the asyncio reactor for Twisted. ([\#12135](matrix-org/synapse#12135)) - Support the stable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440): threads. ([\#12151](matrix-org/synapse#12151)) - Add a new Jinja2 template filter to extract the local part of an email address. ([\#12212](matrix-org/synapse#12212)) Bugfixes -------- - Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse v1.48.0. ([\#12090](matrix-org/synapse#12090)) - Fix a long-standing bug when redacting events with relations. ([\#12113](matrix-org/synapse#12113), [\#12121](matrix-org/synapse#12121), [\#12130](matrix-org/synapse#12130), [\#12189](matrix-org/synapse#12189)) - Fix a bug introduced in Synapse 1.7.2 whereby background updates are never run with the default background batch size. ([\#12157](matrix-org/synapse#12157)) - Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse v1.41.0. ([\#12175](matrix-org/synapse#12175)) - Fix a bug introduced in Synapse 1.54.0 that broke background updates on sqlite homeservers while search was disabled. ([\#12215](matrix-org/synapse#12215)) - Fix a long-standing bug when a `filter` argument with `event_fields` which did not include the `unsigned` field could result in a 500 error on `/sync`. ([\#12234](matrix-org/synapse#12234)) Improved Documentation ---------------------- - Fix complexity checking config example in [Resource Constrained Devices](https://matrix-org.github.io/synapse/v1.54/other/running_synapse_on_single_board_computers.html) docs page. ([\#11998](matrix-org/synapse#11998)) - Improve documentation for demo scripts. ([\#12143](matrix-org/synapse#12143)) - Updates to the Room DAG concepts development document. ([\#12179](matrix-org/synapse#12179)) - Document that the `typing`, `to_device`, `account_data`, `receipts`, and `presence` stream writer can only be used on a single worker. ([\#12196](matrix-org/synapse#12196)) - Document that contributors can sign off privately by email. ([\#12204](matrix-org/synapse#12204)) Deprecations and Removals ------------------------- - **Remove workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. Breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](matrix-org/synapse#11700 - **`synctl` has been moved into into `synapse._scripts` and is exposed as an entry point; see [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved). ([\#12140](matrix-org/synapse#12140)) - Remove backwards compatibilty with pagination tokens from the `/relations` and `/aggregations` endpoints generated from Synapse < v1.52.0. ([\#12138](matrix-org/synapse#12138)) - The groups/communities feature in Synapse has been deprecated. ([\#12200](matrix-org/synapse#12200)) Internal Changes ---------------- - Simplify the `ApplicationService` class' set of public methods related to interest checking. ([\#11915](matrix-org/synapse#11915)) - Add config settings for background update parameters. ([\#11980](matrix-org/synapse#11980)) - Correct type hints for txredis. ([\#12042](matrix-org/synapse#12042)) - Limit the size of `aggregation_key` on annotations. ([\#12101](matrix-org/synapse#12101)) - Add type hints to tests files. ([\#12108](matrix-org/synapse#12108), [\#12146](matrix-org/synapse#12146), [\#12207](matrix-org/synapse#12207), [\#12208](matrix-org/synapse#12208)) - Move scripts to Synapse package and expose as setuptools entry points. ([\#12118](matrix-org/synapse#12118)) - Add support for cancellation to `ReadWriteLock`. ([\#12120](matrix-org/synapse#12120)) - Fix data validation to compare to lists, not sequences. ([\#12128](matrix-org/synapse#12128)) - Fix CI not attaching source distributions and wheels to the GitHub releases. ([\#12131](matrix-org/synapse#12131)) - Remove unused mocks from `test_typing`. ([\#12136](matrix-org/synapse#12136)) - Give `scripts-dev` scripts suffixes for neater CI config. ([\#12137](matrix-org/synapse#12137)) - Move the snapcraft configuration file to `contrib`. ([\#12142](matrix-org/synapse#12142)) - Enable [MSC3030](matrix-org/matrix-spec-proposals#3030) Complement tests in CI. ([\#12144](matrix-org/synapse#12144)) - Enable [MSC2716](matrix-org/matrix-spec-proposals#2716) Complement tests in CI. ([\#12145](matrix-org/synapse#12145)) - Add test for `ObservableDeferred`'s cancellation behaviour. ([\#12149](matrix-org/synapse#12149)) - Use `ParamSpec` in type hints for `synapse.logging.context`. ([\#12150](matrix-org/synapse#12150)) - Prune unused jobs from `tox` config. ([\#12152](matrix-org/synapse#12152)) - Move CI checks out of tox, to facilitate a move to using poetry. ([\#12153](matrix-org/synapse#12153)) - Avoid generating state groups for local out-of-band leaves. ([\#12154](matrix-org/synapse#12154)) - Avoid trying to calculate the state at outlier events. ([\#12155](matrix-org/synapse#12155), [\#12173](matrix-org/synapse#12173), [\#12202](matrix-org/synapse#12202)) - Fix some type annotations. ([\#12156](matrix-org/synapse#12156)) - Add type hints for `ObservableDeferred` attributes. ([\#12159](matrix-org/synapse#12159)) - Use a prebuilt Action for the `tests-done` CI job. ([\#12161](matrix-org/synapse#12161)) - Reduce number of DB queries made during processing of `/sync`. ([\#12163](matrix-org/synapse#12163)) - Add `delay_cancellation` utility function, which behaves like `stop_cancellation` but waits until the original `Deferred` resolves before raising a `CancelledError`. ([\#12180](matrix-org/synapse#12180)) - Retry HTTP replication failures, this should prevent 502's when restarting stateful workers (main, event persisters, stream writers). Contributed by Nick @ Beeper. ([\#12182](matrix-org/synapse#12182)) - Add cancellation support to `@cached` and `@cachedList` decorators. ([\#12183](matrix-org/synapse#12183)) - Remove unused variables. ([\#12187](matrix-org/synapse#12187)) - Add combined test for HTTP pusher and push rule. Contributed by Nick @ Beeper. ([\#12188](matrix-org/synapse#12188)) - Rename `HomeServer.get_tcp_replication` to `get_replication_command_handler`. ([\#12192](matrix-org/synapse#12192)) - Remove some dead code. ([\#12197](matrix-org/synapse#12197)) - Fix a misleading comment in the function `check_event_for_spam`. ([\#12203](matrix-org/synapse#12203)) - Remove unnecessary `pass` statements. ([\#12206](matrix-org/synapse#12206)) - Update the SSO username picker template to comply with SIWA guidelines. ([\#12210](matrix-org/synapse#12210)) - Improve code documentation for the typing stream over replication. ([\#12211](matrix-org/synapse#12211))

clokep force-pushed the clokep/redis-types branch from 8327432 to 28449fd Compare February 18, 2022 21:26

clokep added 2 commits February 22, 2022 14:00

Fix some type hints for txredis.

51112d8

Newsfragment

d248e96

clokep force-pushed the clokep/redis-types branch from 28449fd to d248e96 Compare February 22, 2022 19:04

clokep requested a review from erikjohnston February 22, 2022 21:57

Merge remote-tracking branch 'origin/develop' into clokep/redis-types

44b3810

clokep marked this pull request as ready for review March 7, 2022 13:40

clokep requested a review from a team as a code owner March 7, 2022 13:40

clokep removed the request for review from a team March 7, 2022 13:40

erikjohnston approved these changes Mar 8, 2022

View reviewed changes

Merge remote-tracking branch 'origin/develop' into clokep/redis-types

662c6b8

clokep merged commit d8bab67 into develop Mar 8, 2022

clokep deleted the clokep/redis-types branch March 8, 2022 12:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix type hints for txredis. #12042

Fix type hints for txredis. #12042

clokep commented Feb 18, 2022 •

edited

Loading

clokep commented Feb 18, 2022

clokep commented Feb 22, 2022

erikjohnston commented Feb 23, 2022

clokep commented Feb 23, 2022

erikjohnston commented Feb 24, 2022

clokep commented Mar 7, 2022

Fix type hints for txredis. #12042

Fix type hints for txredis. #12042

Conversation

clokep commented Feb 18, 2022 • edited Loading

clokep commented Feb 18, 2022

clokep commented Feb 22, 2022

erikjohnston commented Feb 23, 2022

clokep commented Feb 23, 2022

erikjohnston commented Feb 24, 2022

clokep commented Mar 7, 2022

clokep commented Feb 18, 2022 •

edited

Loading